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SELECTIVE ENHANCEMENT OF DIGITAL IMAGES 

CROSS REFERENCE TO RELATED APPLICATION 

This application claims the benefit of United States provisional application Serial No. 
60/456,150 filed March 19, 2003, titled "System for Selective Noise Reduction and 
Enhancement of Digital Images," 



Currently available noise reduction processes address noise reduction from a global 
perspective (applying noise reduction to an entire image) often softening the image to an 
undesirable degree. Such problems exist bofli for luminance noise and chrominance noise. 
There are regions in images (such as dark hair and shadows) where luminance noise does not 
distract firom the photographic qualities of the image and are often not perceived as noise. 
Chrominance noise, however, is more visible in the same areas and must be reduced differently. 

Most users of image editing ^plications face difficulties with 'targeting" certain areas in 
an image. For example, a user who wants to sharpen the plant in the foreground of an image, but 
not the sky in the background of the image, faces a challenging task. In common image editing 
apphcations, such as Adobe Photoshop®, the user would have to create a "selection" for the 
plant, before applying an image enhancement filter, for instance, a shaipening filter. Typically, 
the user has to "draw*' the selection using a pointing device, such as a computer mouse, around 
the plant Only after creating such a selection, can the user sharpen the plant 

Further, the user often wants to sharpoi the plant to a high degree and the background to a 
lower degree. To do so,, the user would first have to select the plant, sharpen it to a high degree, 
then select everything else but the plant, and sharpen this to a lower degree. In another example, 
given the case that there is a person in the given image and the user wants to shaipea the plants 
in the image to a high extent, the background to a low extent, and the hair and the skin of the 
person in the hnage to a medium extent, using selections vrith conventional apphcations becomes 
a highly challenging task. 

Selecting an area in an image is a difScult task. Therefore, image editing apphcations such 



BACKGROUND 



It is a well-known problem that noise in digital images is present throughout the image. 
While noise may ^pear more in certain attributes of a digital image, e.g., against sky, skin, 
background, etCy noise may not be as visible when presmt against other detail types. 
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as Adobe Photoshop® offer a variety of different selection methods, all of which have a steep 
learning curve. What is needed is a method and system to make selective enhancement an image 
easier, and which would be applicable for all types of image enhancement filters, such as 
shaipening, noise reduction, contrast changes, conversion to black and white, color enhancement 
etc. Such a method and system would provide for a range of image enhancements on a selective 
basis. Preferably, such a method and system would be able to process a digital image by 
applying an image processing filter as a function of multiple image characteristics, or as a 
function of an image characteristic and the input firom a user pointing device. 

SUMMARY 

The disclosed method and system meets this need by providing for a range of image 
enhancements on a selective basis. The method and system is able to process a digital image by 
applying an image processing filter as a function of multiple target image characteristics, or in a 
further embodunent, as a function of target image characteristic and the mput firom a user input 

■ 

device. 

A method for image processing of a digital image comprismg pixels having charactaistics 
is disclosed, comprising applying an image processing filter as a function of the correspondence 
between each pixel and a first target image characteristic and a second target image 
characteristic. 

A metiiod for image processing of a digital image comprising pixels having characteristics 
is disclosed, comprising the steps of providing an image processing filter, receiving first target 
image characteristics, receiving second target image characteristics, determining for each pixel to 
be processed, the correspondence between the characteristics of that pixel and the first target 
image characteristics and second target image characteristics and processing the digital image by 
applying the image processing filter as a function of the determined correspondence between 
each pixel and the first target image characteristics and second target image characteristics. In 
various embodiments, the image processing filter may be, for example, a noise reduction filter, a 
sharpening filter, or a color change filter. 

In a further embodiment, an adjustment parameter may be received, and then the 
application of the image processing filter is also a function of the adjustment parameter. In 
various embodiments the adjustment parameter may be an opacity parameter or a luminosity 
parameter. 

In still further embodiments a graphic user interface may be provided for receiving the first 
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target image chaiacteiistics, the second target image characteristics, and optionally the 
adjustment parameter. The graphic user interface for receiving the adjustment parameter 
optionally may comprise a slider. 

In various embodiments the first target image characteristics, or the second target image 
characteristics, may be an image coordinate, a color, or an image structure, and indicia may be 
used to represent target image characteristics! 

In a still further embodiment, the graphic user interface comprises a tool to detennine the 
pixel characteristics of an image pixel 

Jn. a fiirflier embodiment, a camera-spedfic defealt settings are provided. 

An ^pUcation program inteifece is disclosed, embodied on a.computer-readable medium 
for execution on a computer for image processing of a digital image, the digital image 
conqjrising pixels having characteristics, conqmsing a first interface to receive first target image 
characteristics, a second interfece to receive second target image characteristics, a third interface 
to receive a &st adjustment parameter corresponding to the first target image characteristics, and 
a fourth interfece to receive a second adjustment parameter corresponding to the second target 
image characteristics. OptionaUy, a fiflh inteifece comprismg indicia representing the first target 
image characteristics, and a sixth interfece comprising indicia representing the second target 
image characteristics, may be added. A tool to detemiine the pixel characteristics of an image 
pixel may also be added to the interface, and optionally, the third interfece and the fourth 
interface may each comprise a slida:. 

A system for image processing of a digital image is disclosed, the digital image comprising 
pixels having characteristics, comprising a processor, a memory in conimunication with the 

* 

processor, and a computer readable medium in communication with the processor, the computer 
readable medium having contents for causing the processor to perform the steps of receiving first 
target image characteristics, receiving second target image characteristics, detemiining for each 
pixel to be processed, the correspondence between the characteristics of that pixel and the first 
target image characteristics and second target image characteristics, and proc^sing the digital 
image by applying the image processing filter as a fimction of the determined correspondence 
between each pixel and the first target image characteristics and second target image 
characteristics. 

Optionally, the computer readable medium further has contents for causing the processor to 
perform the steps of receiving a first adjustinent parameter corresponding to the first target image 
characteristics and receiving a second adjustmetjt parameter corresponding to the second target 
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image characteristics. In a further embodiment, he system of claim further comprises a set of 
camera-specific defeult instructions embodied on a computer-readable medium for execution on 
. a computer. 

A set of camera-specific defeult instmctions embodied on a computer-readable medium is 
disclosed, for execution on a conqjuter for image processing of a digital image, using one of the 
embodiments of the method of the invention. The set of camera-specific default instructions 
may set the state of the application program interfece. 

A method for image processing of a digital image comprising pixels having characteristics 
is disclosed, comprising applying an image processing filter as a fimction of the coirespondence 
between each pixel, the received target image characteristic, and the input received ftom a user 
pointing device. 

A method for image processing of a digital image comprising pixels having characteristics 
is disclosed, conqjrising the steps of providing an image processing filter, receiving a target 
image characteristic, receiving a coordinate fi:om a user pointing device, determining for each 
pixel to be processed, the correisgpondence between the characteristics of that pixel, fhe target 
image characteristics, and the received coordioates, and processing the digital image by applying 
the image processing filter as a function of the determined correspondence between each pixel, 
the target image characteristic, and the received coordinates. In various embodiments the image 
processing filter may be, for example, a noise reduction filter, a sharpening fUter, or a color 
change filter. A graphic user interface for receiving the target image characteristic may be used, 
and optionally tiie graphic user interface may comprise indicia representing the target image 
characteristic. Example target image characteristics include an image coordinate, a color, or an 
image structure. 

An application program interface embodied on a computer-readable medium for execution 
on a computer for image processing of a digital image is disclosed, the digital image comprising 
pixels having characteristics, comprising a first interface to receive a target image characteristic; 
and a second interface to receive a coordinate from a user pointing device. 

A system for image processing of a digital image is disclosed, the digital image comprising 
pixels having characteristics, conqdsiog a processor, a memory in conmiunication with tiie 
processor, a user pointing device, and a computer readable medium in communication with the 
processor, the computer readable medium having contents for causing the processor to perform 
the steps of receiving a target image characteristic, receiving coordinates from a user pointing 
device, determining for each pixel to be processed, the correspondence between the 
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characteristics of that pixel, the taiget image characteristics, and the received coordinates, and 
processing the digital image by ^plying the image processing filter as a function of the 
determined correspondence between each pixel, the target image characteristic and received 
coordinates. 

DRAWINGS 

These and other features, aspects, and advantages of the present mvention will become 
better understood with reference to the following description, fended claims, and 
accompanying drawings, where: 

Figure 1 is a depiction one embodiment of an application user interface suitable for use 
according to the invention. 

Figure 2 is a depiction another embodiment of an ^hcation user interfece suitable for use 
according to the invention. 

Figure 3 is a depiction one embodiment of an application user interface suitable for use 
according to a further embodiment of tiie invention. 

■ 

Figure 4 is a depiction of a user interface showing supplication of the invention. 
Figure 5 is a pictorial diagram of components usable with the . system for enhancing digital 
images according to the present invention. 

Figure 6 is a pictorial diagram of the image sources useable for acquiring a digital image to 
be enhanced according to the present invention. 

Figure 7 is a block diagram of an embodiment of the method of the invention. 
Figure 8 is a block diagram of a further embodiment of the method of the invention. 
Figure 9 is a block diagram of an embodiment of the system of the invention. 
Figure. 10 is a block diagram of a further embodiment of the system of the invention. 

DETAILED DESOEUmON 

The method and program interface of the present invention is useable as a plug-in 
supplemental program, as an independent module that may be integrated into any commercially 
available image processing program such as Adobe Photoshop®, or into any image processing 
device that is capable of modifying and displaying an image, such as a color copier or a self 
service photo print kiosk, as a dynamic library file or similar module that may be inaplemented 
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into other software programs whereby image measurement and modification may be usefiil, or as 
a stand alone software program. These are all examples, without limitation, of image processing 
of a digital image. Although embodiments of the invention which adjust color, contrast, noise 
reduction, and sharpening are described, flie present invention is usefiil for altering any attribute 
or feature of the digital image. 

Furthermore, it will become clear with regard to the current invention that the user 
interface for the current invention may have various embodiments, which will become clear later 
in this disclosure. 

The present invention is also useable with a mefliod and system incorporating 
definable image reference points, as disclosed in U.S. Pub. No. US 2003-009941 1 Al, Ser. No. 
10/280,897. for '-User Definable Image Reference Points", which disclosure is expresdy 
incorporated herein by referaice. 

The Application Program Interface 

The present invention, in ils various embodiments, permits the selection of areas of a 
digital image for enhancement. In preferred embodiments, a user interface conqionent is present 
Those skilled in the art wiU find tiiat multiple methods or implementations of a user interface are 
useful with regard to the current invention. 

In one preferred embodiment of a user interface useable with the present invention, the 
interface aUows tiie user to set a variety of types of image modifications in an image, which can 
be shown as graphic sHders. as shown in Figure 1 . ITie sliders could be implemented in a 
window which floats above the image, as will be evident to tiiose skilled in tiie art with reference 
to this disclosure, hi one preferred embodiment, witii reference to Figure 2. tiie sliders are 
implemented in a window containing zoom enabled previews of the image, before and after 
application of tiie image enhancement. In the embodiment shown in Figure 2. a plurality of 
sHders are available, so tiiat flie chosen image enhancement can operate as a fimction of tiiese 
multiple inputs. 

In another embodiment, witii reference to Figure 3, aplurahty of image characteristics are 
Usted. and tiie user may choose to apply tiie chosen image enhancement (noise reduction in tiie 
case of Figure 3) to flie area sdected. For example, by choosing "skin" from tiie table menu, tiie 
user can paint on tiie noise reduction filter, and only skin areas will be modified. In tiie optional 
fiulher embodiment shown, erase, fill, and clear operations are available. 

The appHcation program interface is embodied on a computa-readable medimn for 
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execution on a computer for image processing of a digital image. The interface receives the 
characteristics of the image which the user desires to select. Li a further emboditnent, a second 
interface receives an image editing function assigned by the user. 

Selective Enhancement Using a Selective Application Matrix 

5 With reference to Figures 1 and 2, the plurality of sliders and graphic icons are inputs to a 

matrix, which for convenience we can describe as a Selective Apphcation Matrix, abbreviated to 

■ 

SAM As will be evident to those skilled in the art, other types of controllers are also possible as 
inputs to the SAM. There are at least two, and typically five or more, SAM controllers. 

Preferably, the SAM controllers are displayed next to the image, and each SAM controller 

10 is linked to a region in the ioaiage. The regions may be described in a variety of ways. In one 
preferred method the regions are described by image feature; for example, the first SAM 
controller may be linked to sky, and the second may be linked to grass (not shown). 

As is evident from Figures 1 and Figure 2, the SAM controller may have an associated 
numerical input interface to set an adjustment parameter for filter opacity, strength, or other 

1 5 variable. In a preferred embodiment a slider is used, but direct iaput or other interfaces are 

possible. In the previ9us sky/grass example, if the user sets the first SAM controller adjiistment 
parameter to 80% and the second controller is set to 20%, the selected filter will be applied to 
80% strength to the sky and to 20% strength to the grass. If the filter is a sharpening filter, the 
sky would be sharpened to 80% and the grass to 20%. The same would occur for a filter that 

20 increases the saturation, reduces noise, or enhances the contrast. As a further example, the filter 
could be a filter tiiat turns a color image into a black and white image, where the sHders would 
control the tonality in the image, so that in the black and white image the slcy would have an 80% 
tonaUty (dark) and the grass would have a 20% tonality (being bright); 

The SAM may be used for the purposes of noise reduction, image sharpening, or any other 

25 image enhancement, where it is desired to be able to selectively apply the image enhancement. 

With reference to Figure 1, each SAM controller in that embodiment is represented by a set 
of icons and a slider for the adjustment parameter. Each of the SAM controllers is accompanied 
by one or more fields (1 . 1 , 1 .2 and 1 .3) that can represent target image characteristics. In Figure 
1, icon 1.1 represents a color, icon 1.2 represents an image structure, and icon 1.3 holds an image 

30 coordinate. In one embodiment, the color can be a RGB value, a stmcture can be a value derived 
from flie differaace of adjac^t pixels (such as the mean luminosity difference of horizontally 
adjacent pixels, or local wavelet, or Fourier components), and an image coordinate could be an X 
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and a Y coordinate. 

. If the first slider is supposed to be "linked" ^th the sky (how the user creates such a "liBk" 
wiU be described below), thea the color icon 1 . 1 would contain a color that represents the sky 
(saturated blue), the structuie field would contain data that represents the structure of sky (a very 
plain structure), and the coordinate field would represent a location somewhere in the sky (top of 
the image). The same principle applies for the second SAM controUer. which may, for example, 
be linked to the "grass" (green, high detail structure, bottom of image). 

The user can either set these values in icons 1.1 through 1.3 manuaUy (such as by cUcking 
on the icon and then selecting a color or a structure firom a palette, or by entering the value via 
the keyboard), or the user can use the eyedropper (see icon 1.5 in Figure 1). Once the user dlicks 
on the eyedropper. he can then cUck in the image. Once he clicks in the image, the software will 
then read the color, structure and the coordinate, and fill these values into the icons 1.1 to 1.3. 
OptionaUy, as shown a checkbox 1.6 can be provided to select or deselect an given SAM 
controller. 

NotaU embodiments require aU of the icons 1.1. 1.2, and 1.3; at least one of them is 
sufficient. For example, in Figure 4, each SAM controller comprises one icon and one slider for 
a parameter adjustment 

Any user control that enables the user to define a value can be used. This could be a field 
where the user can enter a number via the keyboard, a wheel that can be rotated like a volume 
20 control on an ampKfier, or other implementations. 

With reference to Figure 7. a digital image can then be processed using method 10: 

1 1) provide an image processing filter 17; 

12) receive first target image charactCTistics; 

13) receive second target image characteristics; 

14) determine for each pixel to be processed, the correspondence 
between the characteristics 16 of that pixel and the first target image 
characteristics and second target image characteristics; and 

15) process the digital image by applying the image processing filter as a 
function of the detemiined correspondence between each pixel and the 
first target image diaracteristics and second target image characteristics. 

In one embodiment, for each pixel to be processed, the SAM controller whose 
characteristics match the giveapixel best is determined, and using that controller's values as 
inputs for the filter, the pixel is modified. 
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la a further embodiment, a step can be added to receive 19 an adjustment parameter and 
apply the filter 17 as a function of the adjustment parameter. In a still further embodiment, 
camera-specific default settings are provided 21 as described herein. 

For example, where the user wants to sharpen a plant with 80% strength and the sky in the 
5 background with 20% strength, this algorithm would identify some pixels in the image to match 
the characteristics of the SAM controller set to the plant and sharpen those pixels with 80%. 
Other pixels would be identified to match the SAM controller set to the sky and would then be 
sharpened with 20%, and still others mi^t not identify with either and might not be sharpened. 
In order to avoid harsh transitions, definable image reference points could be used to allow 
10 for soft transitions from one area to another, as disclosed in U.S. Pub. No. US 2003-009941 1 Al, 
Ser.No. 10/280,897, for **User Definable Image Reference Points." (That disclosure is 
expressly incorporated herein.) This would be preferred for filters that change luminosity or 
color, as the soft transitions provide a higjier image quality. In filters such as noise reduction or 
sharpening, speed of processing may be more important. 
15 The SAM can be used in many different ways. The filt^ can be any image enhancement, 

and the values of the adjustment parameter can be any dominant parameter of that fiDlter. The 
filters can be color enhancement, noise reduction, sharpening, blmiing, or other filter, and the 
values of the adjustment parameter can control the opacity, the saturation, or the radius used in 
the filter. 

20 In still furtho: embodiments, the filters can be a conversion to black and white or a filter 

that raises the contrast, hi such a filter the user may want to make certain areas a httie darker 
while applying the filter, while brightening other areas. The SAM would then be implemented in 
a way that the value provided for each pixel in the named algorithm is used to darken or liditen 
the pixel to a certain extent 

25 Any filter known in the field of image editing, and any parameter of that filter can be 

controlled by a SAM. 

Calculating A Selective Application Matrix 

As an example of how the appHcation user interface can be used with a filter wiU be 
described. In this embodiment, with reference to Figure 1, the user can chck on one of the icons 
30 representing target image characteristics, such as color icon LI, and redefine tiie color that is 
associated with the associated slider 1 .4. In the following equation, these n colors will be 
referred to as Ci . . .Cn- The setting of a slider (z.e., the desired noise reduction for the color of the 
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slider) wm be referred to as S , . . .S„. fi is preferable to nonnalize . . .S„ so that it ranges from 
0.0 to 1.0, where 1.0 represents 100% noise reduction. 

The desired value Sxy can be calculated for eachpixel in the image as follows: 

Where: 

is the value to be calculated for each pixel in the image I. ranging from MIN to 
to represent for example the opacity of a noise reduction algorithm ^plied. 

n is the amount of sliders that are offered, such as 3 in the given examples. 
m is the amount of target image characteristics that are used in flie process, 
f^is an inversion function, such as V{x) = 1/x, e"^ , l/x^, etc. 
Si is the value of the z-th slider, r^g from MIN to MAX. 

^ ^*-i^dCr.,,arecharacteristicsofapixdorasHder,Qjbeingthej'^cha«^^ " 
1 sHder, Chyj bemg the j* characteristic of the pixel I^. 

The characteristics C can be directly derived from the values received from the target 

image characteristic icons 1.1, 1^. and 1.3 as shown in Figure 1. If the coordinates icon 1.3 is 

provided, the list, of characteristics Q,...Q, will at least include one target image characteristic 

for die horizontal, and one target image characteristic for the vertical coordinate. If a color icon 

1.1 or a structure icon ISL is provided, additional characteristics will be derived from those fields 

Note: To implement a SAM. not all characteristic fields 1.1. 1.2. or 1.3, as sho^vo. in Figure 1. are 
20 required. 

This principle can be used for filters like shaipeiung. noise reduction, color warming and 
other filters where it is desirable to control the opacity of one filter. 

^ "nieSAMcanalsobeused to provide advanced input parameters to afilter. If afilterF 
has one parameter z that the user may want to vary throughout the image, such as I ^ = 
I"(I,x,y,z), this parameter z can be replaced with in order to vary the effect of the filter F'. 

Such a filter F' could be a blurring effect, and the parameter z could be a radius. In that 
case, the shders would probably reach from 0.0 (MIN) to, for instance, 4.0 (MAX) sc _ 
radius between 0.0 and 4.0. The blurring filter F(I, x. y, S,,) would then blur the pixels ofZ 
miage depending on the variable S^, which varies from pixel to pixeL Wili this technique the 
30 usercanblurtheimagewithdifferentradiiatdiferentaieas. For exanq,le. if there were orily 
two sHders and the user "hnked" one slider to the sIq. and set its value to 3.5. and ff the 
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"linked" the second slider with the face in the foreground and set its value to 0.5, the filter would 
blur the sky with a radius of 3.5, tJie face with a radius of 0.5, and other parts of the image with 
varying radii between 0.5 and 3.5. 

Another example for such a filter F' could be any complex image filter with many 
parameters in addition to such as a conversion to black and white, a relief effect, a painterly 
effect, an increase of contrast, etc. Many of such artistic or photographic filters often create "fall 
off areas" or **blown out areas." A "fall off area" is an area in the image th,at is completely black 

area of zero values) afier the filter is applied, and a **blown out area" is an area that is 
purely white. Neither effect is wanted For instance, if the filter appUes a brightening effect, 
areas that were "almost white" before filtering may easily become pure white after filtering. In 
such case it is desirable that this area be darkened while filtering. This could be done, for 
instance, by setting the lowest possible setting of the n sliders (MIN) to a negative value and the 
highest possible setting of the n sliders (MA3Q.to the same positive value, such as -50 and 50, so « 
that Sxy varies firom -50 to 50 for each pixel on the image. The user could connect one of the 
sfiders to that area that was almost white before filtering, and set the sliders value to below zero. 
The filter F'(I, x, y, z) would then receive a low value for z in this area and therefore lower the 
luminosity in this area while applying the filter. Those skilled iu the art will be famihar with 
how to include z into this process. For example, z may be simply added to the luminosity before 
any fiirther filtering takes place. 

Figure 4 shows a sample use of a SAM implementation used to prevent blown out areas 
during the image editing process. Figure 4 (top) shows the image without the SAM being used 
and Figure 4 (bottom) shows the image with the SAM used to prevent the blown out effect. 

» 

Using the SAM for Camera-Specific Noise Reduction 

% 

« 

The SAM can be combiaed with camera-specific noise reduction filters to provide 
optimized noise reduction and increased control. If this combination is desired, the 
implementation of the sUders in Figure 1 can be camera specific. For example, a camera with a 
uniform noise behavior may require fewer sHders (for example n = 3) while a camera that 
produces noise that is more stmcture dependent, relative to other cameras, may require a larger 
number of sUders (for example 7t = 8). 

In a fiirther embodiment of the invention, the default settings of the sUders could be made 
camera-specific. If the camera has a tendency to produce excessive noise in blue areas of an 
image, the SAM might include a slider with a color field, which is set by default to blue and a 
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slider value which is set by default to a high setting. An implementation for a specific camera is 
shown in Figure 2. 

Noise and Detail Specific Tools 

The use of detail-specific noise reduction and detail enhancement tools are provided in one 
5 embodiment of the current invention allowing xisers to use conventional pointing devices, such as 
a computer mouse or a pressure sensitive graphics tablet and pen, to apply the presrabed tool. 
Current applications only allow uscts to brush-in effects in an image such as a fixed color, a 
darkening or a lightening effect, a sharpening or a blurring effect 

With reference to Figure 3, one embodiment of the current invention provides detail 
10 specific filters that focus on individual types of detail in order to protect specific details in flie 
noise reduction process. By focusing on specific details that occur in most images, a specific 
process can be created for selective noise reduction that considers specific detail types. A variety 
of detail specific noise reducers can be designed, such as one designed for sky details, 
background details, skin details, and shadow details, for example. The noise reduction filter (in 
1 5 other embodiments other filters could be used) can then be brushed-in using a user pointing 
device 36. 

With reference to Figure 8, a digital image can then be processed by method 20: 
ir) provide an image processing filter 17'; 

■ 

12') receive a target image characteristic; 
20 1 8) receive a coordinate firom a user poiutiag device 36; 

14') determine for each pixel to be processed, the correspondence between the 
characteristics 16' of that pixel, the target image characteristic, and the received coordinates. 

15') process the digital image by applying the image processing filter 17' as a function of 
the determined correspondence between each pixel, the target image characteristic, and the 
25 received coordinates. 

Creating Noise Brashes for Different Image Structures and Details 

In order to create a detail-specific noise reduction filter, a general noise reduction 
algorithm is required which differentiates between chrominance and Imninance and different 
frequencies. For example, a filter could have one parameter for small noise, for noise of 
30 intermediate sizes, and for large noise. If a filter based on a Laplace pyramid, Wavelets, or 
Fourier analysis is used, those skilled in the art will know how to create a noise reduction filter 
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that differentiates between various frequencies/bands. The filter may also accq}t dififerent 
parameters for the luminance noise reduction strength versus chrominance noise reduction 
strength. If this is done, the filter will be able to accept a few different paramciers: 



Table 1 



High Frequencies / Luminance 


Medium Freq. / Luminance 


Low Freq. / Luminance 


High Freq. / Chrominance 


Medium Freq. / Chrominance 


Low Freq. / Chrominance 



For best results, locate a suitable combination of such parameters. 

It is possible to correlate these target image characteristics to specific enhancement 
algorithms using heuristic methods. For example, using a plurality of images, select one image 
structure type, sudi as sky, skin, or background. Using trial and error, experimyeait with different 
values for the noise reducer on all of the images to determine the optimal combination for the 
noise reduction for this structure type. For example, for the stmcture type bad^ound, the 
following-parameters might be suitable: 



Table2 



100% 


100% 


100% 


100% 


100% 


100% 



Since the background of an image is typically out-of-focus and therefore blurry, it is 
acceptable to reduce both chrominance and Ixrminance noise to a strong degree. On the other 
hand, the structxire type sky migjit have the following parameters: 



Table 3 



25% 


50% 


75% 


100% 


100% 


100% 



This combination would be suitable as sky often contains very fine cloud details. To 
maintain these details, the first table entry (high frequencies/luminance) is set to 25% only. 
However, as sky consists mostly of very large areas, it is important that the low frequencies are 
reduced to a rather large extent, so that the sky does not contain any large irregularities. Because 
of this, the third table entry is set to 75%. The lower three table entries, which cover the 
chrominance noise, are all set to 100%, as sky has a rather uniformly blue color, against which 
color irregularities can be seen very well. 

Treating Chrominance and Luminance Noise 

One embodiment of the current invention provides a range of options for optimally 
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reducing chrominance noise (noise that consists of some degree of color) and luminance 
(noise with no appearance of color) in a digital image. The system described employs a range of 
techniques while using an ^proach that spHts the image into one luminance chaimel (CI) and 
two chrominance channels (C2 and C3). The process of sphtting the chrominance information 
5 from the luminance information in the image may be performed in a constant fashion or using a 
camera-deppndent implementaition. 

Splitting the image in Chrominance and Luminance 

To gain the channels Ci, C2, and C3, the image can be transformed either into "Lab" or 
"YCrCb" mode, or in an individual fashion, where Ci could be calculated as xir + X2g + xab, all x 
10 being positive. While doing so, it is important that a set of X1...X3 is found which leads to a 
channel Ci that contains the least possible chrominance noise. To do so, take an image 
containing a significant amount of chrominance noise and find a set of X1...X3 where the 
grayscale image Ci has the least noise. Finding the set of xi . . .X3 with trial and error is an 
appropriate approach. To obtain the image channels C2 and C3, two fiirther triples of numbers 
yi . . .y3 and zi . . .Z3 are required, where all three sets most be hnear independent. If the matrix [x, 
y, z] were linear dependent it would not be possible to regain the original image colors out of the 
information Q...C3 after the noise reduction were performed. Find values for yi.. .yj and zi.. .Z3 
so that the resulting channeb Q and C3 contain the least luminance information (flie image 
should not look like a grayscale version of the original) and the most chrominance noise (the 
20 color structines of the original should manifest themselves as a grayscale pattern of maximal 
contiast in tiie cannels C2 and C3). The two tiiples (-1,1,0) and (0,-1,-1) are good values to start 
with. If the user interface or system involves a step that requests information from the user on 
what digital camera / .digital chip / recording process is used, it may preferable to adjust the three 
triples xi . . .X3 . . . zi . . .Z3 based on the camera. If a camera produces a predominant amount of 
noise in the blue channel, it may be preferable to set X3 to a low value. If it has the most noise in 
the red channel, for instance with multiple-sensor-per-pixel chips, it may make saise to set xi< 
X3. 

System 

Preferably, the invention will be embodied in a computer program (not shown) either by 
30 coding in a high level language, or by preparing a filter which is compUed and available 

adjunct to an image processing program. For example, in a preferred embodiment, the SAM 



as an 
is 
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compiled into a plug-in filter that caa operate within third party image processing programs, such 
as Photoshop®. It could also be implemented in a stand alone program, or in hardware, such as 
digital cameras. 

Aay currently existing or future developed computer readable medium suitable for storing 
5 data can be used to store the programs embodying the afore-described methods and algorithms, 
including, but not limited to hard drives, floppy disks, digital tape, flash cards, compact discs, 
and DVDs. The computer readable medium can comprise more than one device, such as two 
linked hard drives. This invCTtion is not limited to the particular hardware used herein, and any 
hardware presently existing or developed in the future that permits image processing can be 
10 used. 

With reference to Figure 9, one embodiment of a system 100 of the present invention 
comprises a processor 1 02, a mCTiory 1 04 in communication with the processor 1 02/ and a 
computer readable mediimi 106 in communication with the processor 102, having contents for 
causing the processor 102 to perform the steps of one of the embodimmts of the method 10 of 

15 Figure 7. With reference to Figure 10, a further embodiment of a system 200 of the present 

invention comprises a processor 102, a memory 104 in communication with the processor 102, a 
user pointing device 36, and a computer readable medium 106 in communication with the 
processor 102, having contents for causing the processor 102 to perform the steps of one of the 
embodiments of the method 20 of Figure 8. 

20 With reference to Figure 5 and Figure 6, one hardware configuration useable to practice 

various embodiments of the method of the invention comprises a con^)uter monitor 32 and 
computer CPU 34 comprising processor 102 and memory 104, program instructions on computer 
readable medium 106 for executing one of the embodiments of method 10 or method 20 on a 
digital image 38, for output on one or more than one printer type 42, or a digital display device 

25 30 through the Intemet. In at least one embodiment a userpointiQg device 36 provides 

coordiuate information to CPU 34. Various pointing devices could be used, including pens, 
mice, etc. As will be evident to those skilled in the art with reference to this disclosure, various 
combinations of printer type 42 or digital display device 30 wUl be possible. 

Digital image 38 could be obtained &om various image sources 52, including but not 

30 limited to film 54 scaimed through a film scanner 56, a digital camera 58, or a hard image 60 
scanned througji an image scaimer 62. It would be possible to combine various components, for 
example, integrating computer monitor 32 and computer CPU 34 with digital camera 58, film 
scaimer 56, or image scanner 62. 
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In one embodiment, it is possible to have the program instructions query the components 
of the system, including but not limited to any image processing program bemg used, or printer 
being used, to deteraiine default settings for such programs and devices, and use those 
parameters as the inputs into the SAM. These parameters may automaticaUy be determined 
without operator intervention, and set as the defaults for the system. Depending upon the 
particular needs, these defaults may be further changeable by operator intervention, or not 

It is to be understood that in this disclosure a reference to receiving parameters includes 
such automated receiving means and is not to be Limited to receiving by operator input The 
receiving of parameters will therefore be accomplished by a module, which may be a 
combination of software and hardware, to receive the parameters either by operator input, by 
way of example through a digital display device 32 interfece, by automatic detennination of 
defeults as described, or by a combination. 

The enhanced digital image is fben stored in a memory block in a data storage device 
within computer C3>U 34 and may be printed on one or more pxmters, transmitted over the 
Internet, or stored for later printing. 

In the foregoing specification, the invention has been described with refeience to specific 
embodiments thereof. It will, however, be evident that various modifications and change may 
be made thereto without departing fi»m the broader spirit and scope of the invention. The 
specification and drawing are, accordingly, to be regarded in an illustrative rather than a 
restrictive sense. It should be appreciated that flie present invention should not be construed as 
limited by such embodiments, but rather constraed according to tiie below claims. 

All features disclosed in the specification, including tbe claims, abstract and drawings, and 
all the steps in any method or process disclosed, maybe combined in any combination, except 
combinations where at least some of such features and/or steps are mutiially exclusive. Each 
feature disclosed in the specification, including the claims, abstract, and drawings, can be 
replaced by alternative features serving the same, equivalent or similar purpose, unless ejq)ressly 
stated otherwise. Ih.us, unless expressly stated otherwise, each featiire disclosed is one example 
only of a generic series of equivalent or similar features. 

This inventirai is not limited to particular hardware described herein, and any hardware 
presentiy existing or developed in the future that permits processing of digital images using the 
method disclosed can be used, including for example, a digital camera system. 

Any currentiy existing or future developed conq)uter readable medium suitable for storing 
data can be used, including, but not limited to hard drives, floppy disks, digital tape, flash cards. 
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compact discs, and DVDs. The computer readable medium can comprise more than one device, 
such as two linked hard drives, in communication with the processor. 

Also, any elraient in a claim that does not explicitly state "means for" performing a 
specified function or "step for"* performing a specified fimction, should not be interpreted as a 
"means" or "step" clause as specified in 35 U.S.C. § 1 12. 

It will also be understood that the term "comprises" (or it grammatical variants) as used in 
this specification is equivalent to the term **includes" and should not be taken as excluding the 
preseuce of other elements or features. 



